Types Of Machine Learning Models Research Articles

Analysis of machine learning prediction reliability based on sampling distance evaluation with feature decorrelation

Despite successful use in a wide variety of disciplines for data analysis and prediction, machine learning (ML) methods suffer from a lack of understanding of the reliability of predictions due to the lack of transparency and black-box nature of ML models. In materials science and other fields, typical ML model results include a significant number of low-quality predictions. This problem is known to be particularly acute for target systems which differ significantly from the data used for ML model training. However, to date, a general method for uncertainty quantification (UQ) of ML predictions has not been available. Focusing on the intuitive and computationally efficient similarity-based UQ, we show that a simple metric based on Euclidean feature space distance and sampling density together with the decorrelation of the features using Gram–Schmidt orthogonalization allows effective separation of the accurately predicted data points from data points with poor prediction accuracy. To demonstrate the generality of the method, we apply it to support vector regression models for various small data sets in materials science and other fields. We also show that this metric is a more effective UQ tool than the standard approach of using the average distance of k nearest neighbors (k = 1–10) in features space for similarity evaluation. Our method is computationally simple, can be used with any ML learning method and enables analysis of the sources of the ML prediction errors. Therefore, it is suitable for use as a standard technique for the estimation of ML prediction reliability for small data sets and as a tool for data set design.

End-to-End Ultrasonic Hand Gesture Recognition

As the number of electronic gadgets in our daily lives is increasing and most of them require some kind of human interaction, this demands innovative, convenient input methods. There are limitations to state-of-the-art (SotA) ultrasound-based hand gesture recognition (HGR) systems in terms of robustness and accuracy. This research presents a novel machine learning (ML)-based end-to-end solution for hand gesture recognition with low-cost micro-electromechanical (MEMS) system ultrasonic transducers. In contrast to prior methods, our ML model processes the raw echo samples directly instead of using pre-processed data. Consequently, the processing flow presented in this work leaves it to the ML model to extract the important information from the echo data. The success of this approach is demonstrated as follows. Four MEMS ultrasonic transducers are placed in three different geometrical arrangements. For each arrangement, different types of ML models are optimized and benchmarked on datasets acquired with the presented custom hardware (HW): convolutional neural networks (CNNs), gated recurrent units (GRUs), long short-term memory (LSTM), vision transformer (ViT), and cross-attention multi-scale vision transformer (CrossViT). The three last-mentioned ML models reached more than 88% accuracy. The most important innovation described in this research paper is that we were able to demonstrate that little pre-processing is necessary to obtain high accuracy in ultrasonic HGR for several arrangements of cost-effective and low-power MEMS ultrasonic transducer arrays. Even the computationally intensive Fourier transform can be omitted. The presented approach is further compared to HGR systems using other sensor types such as vision, WiFi, radar, and state-of-the-art ultrasound-based HGR systems. Direct processing of the sensor signals by a compact model makes ultrasonic hand gesture recognition a true low-cost and power-efficient input method.

Machine learning approach for predicting post-intubation hemodynamic instability (PIHI) index values: towards enhanced perioperative anesthesia quality and safety

BackgroundAdequate preoperative evaluation of the post-intubation hemodynamic instability (PIHI) is crucial for accurate risk assessment and efficient anesthesia management. However, the incorporation of this evaluation within a predictive framework have been insufficiently addressed and executed. This study aims to developed a machine learning approach for preoperatively and precisely predicting the PIHI index values.MethodsIn this retrospective study, the valid features were collected from 23,305 adult surgical patients at Peking Union Medical College Hospital between 2012 and 2020. Three hemodynamic response sequences including systolic pressure, diastolic pressure and heart rate, were utilized to design the post-intubation hemodynamic instability (PIHI) index by computing the integrated coefficient of variation (ICV) values. Different types of machine learning models were constructed to predict the ICV values, leveraging preoperative patient information and initiatory drug infusion. The models were trained and cross-validated based on balanced data using the SMOTETomek technique, and their performance was evaluated according to the mean absolute error (MAE), root mean square error (RMSE), mean absolute percentage error (MAPE) and R-squared index (R2).ResultsThe ICV values were proved to be consistent with the anesthetists’ ratings with Spearman correlation coefficient of 0.877 (P < 0.001), affirming its capability to effectively capture the PIHI variations. The extra tree regression model outperformed the other models in predicting the ICV values with the smallest MAE (0.0512, 95% CI: 0.0511–0.0513), RMSE (0.0792, 95% CI: 0.0790–0.0794), and MAPE (0.2086, 95% CI: 0.2077–0.2095) and the largest R2 (0.9047, 95% CI: 0.9043–0.9052). It was found that the features of age and preoperative hemodynamic status were the most important features for accurately predicting the ICV values.ConclusionsOur results demonstrate the potential of the machine learning approach in predicting PIHI index values, thereby preoperatively informing anesthetists the possible anesthetic risk and enabling the implementation of individualized and precise anesthesia interventions.

Application of metaheuristic algorithms for compressive strength prediction of steel fiber reinforced concrete exposed to high temperatures

The demand for steel fiber-reinforced concrete (SFRC) in construction has surged, particularly due to its enhanced fire resistance, leading to extensive research on its residual properties after elevated temperature exposure. However, conducting experimental tests can be a time-consuming process, demanding significant resources in terms of time, cost, and human resources. In contrast, machine learning (ML) models can rapidly simulate outcomes, enabling researchers to explore a wide range of scenarios efficiently. The previous literature studies used either ensemble or individual models; however, this study made a unique approach to utilize hybrid models, which are more precise compared to the other types of ML models. Accordingly, a data-rich framework containing 304 data samples was utilized in this study to develop an efficient and robust predictive model for the compressive strength (CS) of SFRC at higher temperatures. Support vector regression (SVR) in combination with three distinct optimization algorithms, specifically, particle swarm optimization (PSO), the firefly algorithm (FFA), and grey wolf optimization (GWO) were utilized to develop the hybrid models. In addition, typical ML techniques such as random forest (RF) and decision tree (DT) were used for comparative analysis. Among the five models, the SVR-FFA hybrid model showcased superior predictive accuracy, with the SVR-GWO model following closely. The correlation coefficients (R) for both models exceeded 0.99. The SVR-FFA model demonstrated relative root mean square error (RMSE) and mean absolute error (MAE) values of 0.0375 and 0.0248, respectively, while the SVR-GWO model exhibited corresponding values of 6.0560 and 5.1326. The SHapley Additive exPlanation (SHAP) technique revealed that the influence of temperature is more significant compared to the heating rate. Moreover, a considerable enhancement was observed in CS as the steel fiber volume fraction (Vf) reached 1.5%; on the other hand, no further improvement was noticed when Vf exceeded the 1.5% threshold. The proposed hybrid models are robust and accurate methods to estimate the CS of SFRC in field applications.

Cryptocurrency price prediction based on Xgboost, LightGBM and BNN

The valuation and prediction of cryptocurrency prices have become increasingly important in the financial market. Therefore, this study aims to focus on the selection and evaluation of machine learning models for cryptocurrency valuation. Thus, two types of machine learning models, gradient boosting trees (Xgboost and LightGBM) and neural networks, are compared to determine their effectiveness in generating features for cryptocurrency valuation. Additionally, correlation tests are conducted to identify the most suitable input variables for the models. The results demonstrate that the generated features have a significant impact on the accuracy of machine learning predictions for cryptocurrency prices. It highlights the potential of machine learning models in accurately predicting and evaluating the value of cryptocurrencies. Overall, the findings of this study contribute to the understanding of the role of machine learning in cryptocurrency valuation and provide valuable insights for investors and researchers. By leveraging machine learning techniques, investors can make informed decisions and develop effective investment strategies in the cryptocurrency market. This study contributes to cryptocurrency valuation research. Leveraging machine learning enables informed decisions and effective investment strategies. Furthermore, the findings inform the development of advanced machine learning models and algorithms for cryptocurrency valuation.

Exploring the application and challenges of fNIRS technology in early detection of Parkinson's disease.

Parkinson's disease (PD) is a prevalent neurodegenerative disorder that significantly benefits from early diagnosis for effective disease management and intervention. Despite advancements in medical technology, there remains a critical gap in the early and non-invasive detection of PD. Current diagnostic methods are often invasive, expensive, or late in identifying the disease, leading to missed opportunities for early intervention. The goal of this study is to explore the efficiency and accuracy of combining fNIRS technology with machine learning algorithms in diagnosing early-stage PD patients and to evaluate the feasibility of this approach in clinical practice. Using an ETG-4000 type near-infrared brain function imaging instrument, data was collected from 120 PD patients and 60 healthy controls. This cross-sectional study employed a multi-channel mode to monitor cerebral blood oxygen changes. The collected data were processed using a general linear model and β values were extracted. Subsequently, four types of machine learning models were developed for analysis: Support vector machine (SVM), K-nearest neighbors (K-NN), random forest (RF), and logistic regression (LR). Additionally, SHapley Additive exPlanations (SHAP) technology was applied to enhance model interpretability. The SVM model demonstrated higher accuracy in differentiating between PD patients and control group (accuracy of 85%, f1 score of 0.85, and an area under the ROC curve of 0.95). SHAP analysis identified the four most contributory channels (CH) as CH01, CH04, CH05, and CH08. The model based on the SVM algorithm exhibited good diagnostic performance in the early detection of PD patients. Future early diagnosis of PD should focus on the Frontopolar Cortex (FPC) region.

Large language models for science and medicine.

Large language models (LLMs) are a type of machine learning model that learn statistical patterns over text, such as predicting the next words in a sequence of text. Both general purpose and task-specific LLMs have demonstrated potential across diverse applications. Science and medicine have many data types that are highly suitable for LLMs, such as scientific texts (publications, patents and textbooks), electronic medical records, large databases of DNA and protein sequences and chemical compounds. Carefully validated systems that can understand and reason across all these modalities may maximize benefits. Despite the inevitable limitations and caveats of any new technology and some uncertainties specific to LLMs, LLMs have the potential to be transformative in science and medicine.

Incorporating Context into BIM-Derived Data—Leveraging Graph Neural Networks for Building Element Classification

The application of machine learning (ML) for the automatic classification of building elements is a powerful technique for ensuring information integrity in building information models (BIMs). Previous work has demonstrated the favorable performance of such models on classification tasks using geometric information. This research explores the hypothesis that incorporating contextual information into the ML models can improve classification accuracy. To test this, we created a graph data structure where each building element is represented as a node assigned with basic geometric information. The connections between the graph nodes (edges) represent the immediate neighbors of that node, capturing the contextual information expressed in the BIM model. We devised a process for extracting graphs from BIM files and used it to construct a graph dataset of over 42,000 building elements and used the data to train several types of ML models. We compared the classification results of models that rely only on geometry, to graph neural networks (GNNs) that leverage contextual information. This work demonstrates that graph-based models for building element classification generally outperform classic ML models. Furthermore, dividing the graphs that represent complete buildings into smaller subgraphs further improves classification accuracy. These results underscore the potential of leveraging contextual information via graphs for advancing ML capabilities in the BIM environment.

Predicting preterm birth using auto-ML frameworks: a large observational study using electronic inpatient discharge data.

To develop and compare different AutoML frameworks and machine learning models to predict premature birth. The study used a large electronic medical record database to include 715,962 participants who had the principal diagnosis code of childbirth. Three Automatic Machine Learning (AutoML) were used to construct machine learning models including tree-based models, ensembled models, and deep neural networks on the training sample (N = 536,971). The area under the curve (AUC) and training times were used to assess the performance of the prediction models, and feature importance was computed via permutation-shuffling. The H2O AutoML framework had the highest median AUC of 0.846, followed by AutoGluon (median AUC: 0.840) and Auto-sklearn (median AUC: 0.820), and the median training time was the lowest for H2O AutoML (0.14 min), followed by AutoGluon (0.16 min) and Auto-sklearn (4.33 min). Among different types of machine learning models, the Gradient Boosting Machines (GBM) or Extreme Gradient Boosting (XGBoost), stacked ensemble, and random forrest models had better predictive performance, with median AUC scores being 0.846, 0.846, and 0.842, respectively. Important features related to preterm birth included premature rupture of membrane (PROM), incompetent cervix, occupation, and preeclampsia. Our study highlights the potential of machine learning models in predicting the risk of preterm birth using readily available electronic medical record data, which have significant implications for improving prenatal care and outcomes.

Machine learning-based optimal distributed generation and electric vehicle load management

The load profile of radial distribution networks (RDNs) is significantly impacted when plug-in electric vehicles (PEVs) are connected to them in large numbers. The disturbances in the load profile may lead to increased power losses in distribution lines and deterioration of the network voltage profile. The provision of distributed generation at strategic locations in the distribution network can help to compensate for the impact on the electrical network due to PEV loads. This paper proposes the use of machine learning (ML)-based models for determining the optimal location of distributed generators (DGs) in an RDN. The proposed method considered time-varying load in addition to PEV load. The suggested method determines optimal placement of DGs based on the power loss reduction index and voltage deviation index reduction index. Four distinct types of ML models were used in the proposed approach using synthesised data on the Institute of Electrical and Electronics Engineers' 33-bus RDN. The performance of the ML models was evaluated with respect to mean squared error and mean absolute percentage error and, for the ML models considered, the random forest ML model gave the best performance.

Machine Learning Models for Predicting Zirconocene Properties and Barriers.

Zr metallocenes have significant potential to be highly tunable polyethylene catalysts through modification of the aromatic ligand framework. Here we report the development of multiple machine learning models using a large library (>700 systems) of DFT-calculated zirconocene properties and barriers for ethylene polymerization. We show that very accurate machine learning models are possible for HOMO-LUMO gaps of precatalysts but the performance significantly depends on the machine learning algorithm and type of featurization, such as fingerprints, Coulomb matrices, smooth overlap of atomic positions, or persistence images. Surprisingly, the description of the bonding hapticity, the number of direct connections between Zr and the ligand aromatic carbons, only has a moderate influence on the performance of most models. Despite robust models for HOMO-LUMO gaps, these types of machine learning models based on structure connectivity type features perform poorly in predicting ethylene migratory insertion barrier heights. Therefore, we developed several relatively robust and accurate machine learning models for barrier heights that are based on quantum-chemical descriptors (QCDs). The quantitative accuracy of these models depends on which potential energy surface structure QCDs were harvested from. This revealed a Hammett-type principle to naturally emerge showing that QCDs from the π-coordination complexes provide much better descriptions of the transition states than other potential-energy structures. Feature importance analysis of the QCDs provides several fundamental principles that influence zirconocene catalyst reactivity.

Beyond geofencing: Behavior detection using AIS

This research paper proposes two different methods for removing biases from geo-spatial trajectories obtained from AIS, enabling the creation of machine learning models that generalize well to areas with no training data. We focus here on the task of behavior detection such as moored, underway or drifting from AIS. The first method utilizes data augmentation techniques specifically designed for augmenting geo-spatial trajectories that include angled information such as direction, while the second method is based on engineering features that eliminate these geographical biases. These two methods are compared using different types of machine learning models, including random forests, hidden Markov models, LSTMs, and transformer networks.

A fast machine learning dataloader for epigenetic tracks from BigWig files.

We created bigwig-loader, a data-loader for epigenetic profiles from BigWig files that decompresses and processes information for multiple intervals from multiple BigWig files in parallel. This is an access pattern needed to create training batches for typical machine learning models on epigenetics data. Using a new codec, the decompression can be done on a graphical processing unit (GPU) making it fast enough to create the training batches during training, mitigating the need for saving preprocessed training examples to disk. The bigwig-loader installation instructions and source code can be accessed at https://github.com/pfizer-opensource/bigwig-loader.

Using graph neural networks for wall modeling in compressible anisothermal flows

Abstract Compressible anisothermal flows, which are commonly found in industrial settings such as combustion chambers and heat exchangers, are characterized by significant variations in density, viscosity, and heat conductivity with temperature. These variations lead to a strong interaction between the temperature and velocity fields that impacts the near-wall profiles of both quantities. Wall-modeled large-eddy simulations (LESs) rely on a wall model to provide a boundary condition, for example, the shear stress and the heat flux that accurately represents this interaction despite the use of coarse cells near the wall, and thereby achieve a good balance between computational cost and accuracy. In this article, the use of graph neural networks for wall modeling in LES is assessed for compressible anisothermal flow. Graph neural networks are a type of machine learning model that can learn from data and operate directly on complex unstructured meshes. Previous work has shown the effectiveness of graph neural network wall modeling for isothermal incompressible flows. This article develops the graph neural network architecture and training to extend their applicability to compressible anisothermal flows. The model is trained and tested a priori using a database of both incompressible isothermal and compressible anisothermal flows. The model is finally tested a posteriori for the wall-modeled LES of a channel flow and a turbine blade, both of which were not seen during training.

The hybrid of BERT and deep learning models for Indonesian sentiment analysis

&lt;span&gt;Artificial intelligence (AI) is one example of how data science innovation has advanced quickly in recent years and has greatly improved human existence. Neural networks, which are a type of machine learning model, are a fundamental component of deep learning in the field of AI. Deep learning models can carry out feature extraction and classification tasks in a single design because of their numerous neural network layers. Modern machine learning algorithms have been shown to perform worse than this model on tasks including text classification, audio recognition, imaginary, and pattern recognition. Deep learning models have outperformed AI-based methods in sentiment analysis and other text categorization tasks. Text data can originate from a number of places, including social media. Sentiment analysis is the computational examination of textual expressions of ideas and feelings. This study employs the convolutional neural network (CNN), long-short term memory (LSTM), CNN-LSTM, and LSTM-CNN models in a deep learning framework using bidirectional encoder representations from transformers (BERT) data representation to assess the performance of machine learning. The implementation of the model utilises YouTube discussion data pertaining to political films associated with the Indonesian presidential election of 2024. Confusion metrics, including as accuracy, precision, and recall, are then used to analyse the model’s performance.&lt;/span&gt;

METHODS OF MACHINE LEARNING IN MODERN METROLOGY

In the modern world of scientific and technological progress, the requirements for the accuracy and reliability of measurements are becoming increasingly stringent. The rapid development of machine learning (ML) methods opens up perspectives for improving metrological processes and enhancing the quality of measurements. This article explores the potential application of ML methods in metrology, outlining the main types of ML models in automatic instrument calibration, analysis, and prediction of data. Attention is paid to the development of hybrid approaches that combine ML methods with traditional metrological methods for the optimal solution of complex measurement tasks.

Ponzi scheme detection in smart contracts using the integration of deep learning and formal verification

AbstractBlockchain smart contracts are codes that can execute and enforce rules for blockchain digital transactions. However, smart contracts may contain numerous subtle vulnerabilities, among which Ponzi vulnerabilities are notable. Existing Ponzi scheme contract detection approaches often rely on machine learning models trained on manually extracted features to achieve satisfactory classification results. Nonetheless, the code of a smart contract potentially harbours elusive semantics and characteristics, which compromises the precision and accuracy of vulnerability detection. Therefore, this paper proposes a method of converting operation codes into sequences to process data to avoid losing unnecessary important information, and uses a one‐dimensional convolutional neural network combined with formal verification. This method is named PZ‐C1DZ3(Ponzi‐Conv1D‐Z3) and is used for Ponzi scheme detection. Four types of machine learning models, namely Conv1D, Conv1D‐LSTM, Conv1D‐MLP, and Conv1D‐transformer, are employed for improvement and comparative validation experiments. Additionally, formal verification tool Z3 solver is utilized to conduct formal security verification on the final model, ensuring its safety. Experimental results demonstrate that the improved Conv1D model outperforms other existing models in terms of detection efficiency and accuracy while also meeting the requirements of formal security verification.

Prediction of brain sex from EEG: using large-scale heterogeneous dataset for developing a highly accurate and interpretable ML model

This study presents a comprehensive examination of sex-related differences in resting-state electroencephalogram (EEG) data, leveraging two different types of machine learning models to predict an individual's sex. We utilized data from the Two Decades-Brainclinics Research Archive for Insights in Neurophysiology (TDBRAIN) EEG study, affirming that gender prediction can be attained with noteworthy accuracy. The best performing model achieved an accuracy of 85% and an ROC AUC of 89%, surpassing all prior benchmarks set using EEG data and rivaling the top-tier results derived from fMRI studies. A comparative analysis of LightGBM and Deep Convolutional Neural Network (DCNN) models revealed DCNN's superior performance, attributed to its ability to learn complex spatial-temporal patterns in the EEG data and handle large volumes of data effectively. Despite this, interpretability remained a challenge for the DCNN model. The LightGBM interpretability analysis revealed that the most important EEG features for accurate sex prediction were related to left fronto-central and parietal EEG connectivity. We also showed the role of both low (delta and theta) and high (beta and gamma) activity in the accurate sex prediction. These results, however, have to be approached with caution, because it was obtained from a dataset comprised largely of participants with various mental health conditions, which limits the generalizability of the results and necessitates further validation in future studies. . Overall, the study illuminates the potential of interpretable machine learning for sex prediction, alongside highlighting the importance of considering individual differences in prediction sex from brain activity.

Predictive value of machine learning on fracture risk in osteoporosis: a systematic review and meta-analysis

ObjectivesEarly identification of fracture risk in patients with osteoporosis is essential. Machine learning (ML) has emerged as a promising technique to predict the risk, whereas its predictive performance remains controversial....

Uncertainties of landslide susceptibility prediction: Influences of random errors in landslide conditioning factors and errors reduction by low pass filter method

In the existing landslide susceptibility prediction (LSP) models, the influences of random errors in landslide conditioning factors on LSP are not considered, instead the original conditioning factors are directly taken as the model inputs, which brings uncertainties to LSP results. This study aims to reveal the influence rules of the different proportional random errors in conditioning factors on the LSP uncertainties, and further explore a method which can effectively reduce the random errors in conditioning factors. The original conditioning factors are firstly used to construct original factors-based LSP models, and then different random errors of 5%, 10%, 15% and 20% are added to these original factors for constructing relevant errors-based LSP models. Secondly, low-pass filter-based LSP models are constructed by eliminating the random errors using low-pass filter method. Thirdly, the Ruijin County of China with 370 landslides and 16 conditioning factors are used as study case. Three typical machine learning models, i.e. multilayer perceptron (MLP), support vector machine (SVM) and random forest (RF), are selected as LSP models. Finally, the LSP uncertainties are discussed and results show that: (1) The low-pass filter can effectively reduce the random errors in conditioning factors to decrease the LSP uncertainties. (2) With the proportions of random errors increasing from 5% to 20%, the LSP uncertainty increases continuously. (3) The original factors-based models are feasible for LSP in the absence of more accurate conditioning factors. (4) The influence degrees of two uncertainty issues, machine learning models and different proportions of random errors, on the LSP modeling are large and basically the same. (5) The Shapley values effectively explain the internal mechanism of machine learning model predicting landslide susceptibility. In conclusion, greater proportion of random errors in conditioning factors results in higher LSP uncertainty, and low-pass filter can effectively reduce these random errors.